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Abstract 

In the body of literature on listening strategies to EFL learners, what seems to be lacking is that the focus is on teaching 
listening strategies to learners with little attention to their listening comprehension problems. No local research has been 
conducted on the nature of the Iranian tertiary level students' EFL listening comprehension problems or strategies. 
Therefore, no instrument is available to investigate these constructs. This paper reports the findings of a study that made 
an attempt to develop and test an instrument that will aid researchers identify students’ specific listening problems and 
listening strategy repertoire. The instrument was developed by integrating and validating the available instruments in 
the related literature. The two developed questionnaires were: the Listening Comprehension Problems Questionnaire 
(LCPQ) and the Listening Strategy Use Questionnaire (LSUQ). Problems related to designing and testing this 
instrument is shared and the modifications made to it are presented. The instrument is expected to be useful for 
researchers interested to study the area of EFL listening in a similar setting. 
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1. Introduction 

Studies on English as a Foreign Language (EFL) listening skills usually focus on teaching listening strategies to 
language learners in order to address their listening problems (Goh, 2000; Hassan, 2000; Liu, 2002). However, it should 
be noted that before we teach strategies to learners, we should understand their specific areas of difficulty as well as the 
strategies that they are already aware of. Unfortunately, research on the Iranian tertiary level students' listening 
comprehension problems and strategies is limited. As a result, no suitable instrument is available in the literature to 
support researchers in investigating these students’ listening difficulties and strategies. 

This paper reports the findings of a study in which two questionnaires, namely, the Listening Comprehension Problems 
Questionnaire (henceforth LCPQ) and the Listening Comprehension Strategy Use Questionnaire (henceforth LSUQ) 
were developed based on the available theory, literature, and previous similar instruments. The next section presents the 
procedure that was followed to develop and test the instrument. 

2. Procedure 

The procedure that was followed in developing and testing the instrument included the following phases: 

1. Reviewing the related literature, 

2. Setting the ground rules for developing the instrument, 

3. Constructing the questionnaires, 

4. Modifying the questionnaires based on the feedback elicited from a panel of experts, 

5. Field-testing the questionnaires on a random sample of the target students, and finally, 

6. Refining the instrument. 

What follows is a report of how the questionnaires were developed and then tested following the phases mentioned 
above. 
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3. Reviewing the literature 

The instrument to be discussed in this paper consists of two questionnaires, the Listening Comprehension Processing 
Problems Questionnaire (henceforth LCPQ) and Listening Comprehension Strategy Use Questionnaire (henceforth 
LSUQ).These questionnaires were developed based on a detailed literature review of similar studies on listening such as 
Goh (2000), Hasan (2000), and Liu (2002), Bacon (1992), Cohen, Oxford, and Chi (2005), Vandergrift (1997, 2003, 
2007) and Vandergrift et al. (2006). The first questionnaire, LCPQ, adapts items from the Survey of the Problems of 
English Language Listening (Liu, 2002) and is based on listening problem categories identified by Goh (2000). 
Similarly, LSUQ, as the second questionnaire is constructed on the basis of several listening strategy inventories by 
Bacon (1992), Vandergrift (1997, 2003), and Cohen, Oxford, and Chi (2005) as well as items adapted from the 
Metacognitive Awareness Listening Questionnaire by Vandergrift et al. (2006). Table 1 shows the item numbers in 
LCPQ and LSUQ that were constructed on the basis of the literature review and adopted or adapted from the existing 
related instruments. 


Table 1. Items of LCPPQ and LSUQ 




Items 



Questionnaire 

Constructed 

Adopted 

Adapted 

Total 

LCPS 

8-9, 14-16, 18-20, 
25,26, 

1-6, 10-13, 17,21- 
24, 30 

7,27-29,31 

31 

LSUQ 

33-36, 38 

10-12, 16-19, 27-31 

1-9, 13-15,20,23- 
26, 32, 37 

38 


The first questionnaire was based on Anderson’s listening theory. According to Anderson (1995, cited in Graham and 
Macaro, 2008, p.748), there are three phases involved in listening comprehension: perceptual processing, parsing, and 
utilization. During perceptual processing, listeners focus on the text, segment the phonemes from the speech stream, and 
keep them in their echoic memory. In the second phase, parsing, listeners match this information in their echoic 
memory with the linguistic information in their long-term memory in order to create meaningful mental representations. 
Finally, in the phase of utilization, listeners relate the information which they have collected in the previous phases to 
their schemata in order to understand what they have just heard (Anderson, 1995). Listening comprehension problems 
refer to the drawbacks that may happen at these three cognitive stages (Goh, 2000). 

As for the listening strategies, Vandergrift (1997) offers a taxonomy of listening strategies that includes metacognitive, 
cognitive, and socio-affective strategies. According to Vandergrift (2003), cognitive strategies involve mental activities 
for using the language to complete a task. When students use known words to guess the meaning of unknown 
vocabulary items in a text, they use a cognitive strategy. Likewise, focusing on some parts of the text while ignoring 
other less important parts is a cognitive strategy. Metacognitive strategies are mental activities that direct learners in 
their language learning process. For instance, when learners become aware of the fact that reading some background 
materials will prepare them to listen more effectively, they employ a metacognitive listening strategy. Another example 
would be learning to focus on the main topic and the keywords while ignoring distracters like noise and other people 
around. Finally, socio-affective strategies refer to activities that involve interaction with other individuals or affective 
control in language learning. Common examples of socio-affective strategies include working with others to solve one’s 
listening problems and learning to relax when feeling afraid of not understanding others. Anderson’s (1995) theory 
along with Vandergrift’s (1997) taxonomy provided the theoretical foundation for the present developmental study. 

4. Setting the ground rules 

Questionnaires are useful instruments that can provide insights into learner awareness of their process of listening (Liu, 
2002; Hassan, 2000; Mareschal, 2007) as well as listening comprehension problems and listening strategies. 
Questionnaires have the advantage of “quickly providing information on a wide variety of language learning variables” 
(Dornyei, 2003 cited in Mareschal, 2007, p.50). Before constructing a questionnaire, it is important to set a number of 
ground rules to ensure the usefulness of the outcome. 

The ground rules based on which the items of the questionnaires were constructed were determined based on the 
literature on listening comprehension in general (Feyton, 1991; Field, 2008a), learners’ listening comprehension 
problems (Goh, 2000; Hasan, 2000; Liu, 2002; Field, 2004), listening strategies (Berne, 2004; Vandergrift, 2007; 
Macaro, Graham, &Vanderplank, 2007), and the related literature on second language research methodology (Dornyei, 
2003; Mackey & Gass, 2005; McKay, 2006, Dornyei, 2007). These rules included: 

1. Clarity: The instructions and items should be clear. Definitions, synonyms, or examples were added 
next to difficult terms. In addition, the translation of each item in Persian was added below its English 
version. The translation was carried out by the first author and validated by two Persian-speaking 
lecturers, one English language lecturer (holding a PhD in TESL) as well as one Persian language 
lecturer. 

2. Economy: The questionnaires should be as concise as possible. As suggested by Dornyei (2003, pp. 
18-19), "long questionnaires can be counterproductive". 
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3. Appropriate layout: The "space economical font" of 11-point Times New Roman should be used and 
placing the “response options next to the questions not below" (Dornyei, 2003, p. 14). 

4. Construct Validity: The domains and/or items of an instrument should be informed by established and 
relevant theory. The listening comprehension problem items were developed according to Anderson's 
(1995) listening theory (which was also followed by Goh, 2000). Additionally, the listening strategy 
items were based on Vandergrift's (1997, 2003) order in his taxonomy; 

5. Internal reliability: This can be improved by removing ambiguous items and/or terms and by avoiding 
double-barreled questions, which raise more than one issue. 

6. Ease of tabulation and interpretation of the responses: A Likert scale should be used in which the 
respondents are required to rate the frequency of encountering a specific listening comprehension 
problem or using a particular listening strategy on a scale of 1 (never) to 5 (always). 

Following these principles, the researchers expected to come up with an instrument that was high in validity, reliability, 
and practicality. 

5. Constructing the questionnaires 

Some items were adopted from the existing questionnaires on listening comprehension problems and strategies if they 
were clear and relevant to the Iranian context. Others were adapted by the researchers to make them more 
comprehensible and more appropriate to the Iranian context. As a result, two item pools of 31 for LCPQ and 38 for 
LSUQ were developed to be validated by a panel of experts discussed in the next section. 

6. Modifying the questionnaires 

While qualitative method was employed in constructing the instrument, both qualitative and quantitative methods were 
used in validating it. After the construction of items based on a literature review and the adoption or adaptation of other 
items from the existing related instruments, the questionnaires were validated through expert judgment and field-test. 
These would help the researchers obtain feedback from both the experts and the students on the validity, reliability, and 
item clarity of the instrument (Dornyei, 2003; Vandergrift et al., 2006; Creswell, 2008). 

The first draft of the instrument was presented or e-mailed to 10 EFL experts, who judged on the “redundancy, content 
validity, clarity, and readability” (Dornyei, 2003 cited in Vandergrift et al., 2006, p. 440) of its items. Seven out of the 
ten experts evaluated the instrument providing several useful comments. Tables 2 and 3 summarize the modifications 
made to the first draft of the instrument based on the experts’ comments. These comments involved rewording, 
rearranging, deleting, as well as adding items, examples, definitions, and/or easier terms. 


Table 2. Expert Comments on LCPQ 


General Area 


Comments 

Details 

Items 28 & 31 are similar. In both, the 
"main ideas" are used. 


Modifications 

In item 31, "main idea" was changed to 
"supporting ideas". 


The words "catch" in item 2, "recall" in They were changed into “understand”, 
item 11, and "subsequent" in item 25 are “remember”, and “later" respectively. 

1 Rewording difficult to understand. 

The word "text" has been used The words "material" in item 16 and 
inconsistently. It was sometimes "input" in items 25 & 29 were reworded as 
replaced by "material" in item 16 or by "text". 

"input" in items 25 & 29. 


The items in the perception section are 
not arranged from general to specific: 
concentration, sounds, and words. 

2 Rearranging ~r Vi 3 iTT 3 3 i 3“ These items were rearranged accordingly. 

b b Item 29 should be moved and replaced b b J 

by item 31 for a better sequence because 

item 31 is more general. 


^ Deleting 

Redundant Words, 


In item 23, the phrase "or concepts" is The phrase "or concepts" was deleted, 
redundant. 
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Phrases, or Items 

Items 5 and 6 are related to speaking 
rather than listening problems. 

Item 7 is not appropriate since Iranian 
teachers are not comparable to native 
models. 

Items 5-7 were deleted. 


Items 9 and 17, 2 and 13, 8 and 15 are 
the same. 

One item was removed from each group. 

Adding Some 

4 Necessary Words 
or Phrases 

Item 30 is incomplete and the phrase "of 
the text" is missing. 

The phrase "both audio and video" 
should be added to the instruction. 

The phrases were added. 

5 Typos 

Items 8, 14, 20, & 30 include errors. 

Typos were corrected. 

Table 3. Expert Comments on LSUQ 


Comments 

1 . r , 

General 

Details 

Modifications 


1 Sentence Some phrases are too specific, including: 

Rewording 1. “unknown words” in items 2-7, The phrases were respectively 

2. “talk shows on the radio” and “TV reworded as: 


shows” in items 13 & 14, and 1- “what 1 

3. “my teacher” in item 38. hear”, 

2. “radio and 
TV”, and 

3. “others”. 


The word "response" is difficult to understand. 

The term was replaced 
"answer". 

by 

Wrong use of “during” in item 20. 

The word “during” 
revised to “before”. 

was 


Deleting 

Redundant 

Words, 

The phrase "in an oral text" in item 5, the words 
"usually" in item 32, “paralinguistics” in item 3, and 
the phrase “right away” from item 27 are redundant. 

These were all deleted. 

Phrases, or 
Items 

Item 32, 33, and 34 are similar 

These were merged into one 
item. 

Adding Some 
^ Necessary 
Words or 
Phrases 

The phrase "both audio and video" should be added 
to the instruction. 

The phrase "other things or people around me", the 
word “mental” (mental summary), and the phrase 
“try to” should be added to items 24, 8, and 27, 
respectively. 

The necessary words and 
phrases were added. 


Example is needed for item 6. 

An example was added. 


4 Typos Items 9 and 16 include errors. 


The typos were corrected. 


Appendices A and B show a detailed account of the original items and their revised versions in LCPQ and LSUQ in the 
order they appear in Tables 2 and 3. As a result of expert judgment, the LCPQ and LSUQ were reduced to 24 and 35 
items, respectively. 

7. Field-testing the questionnaires 

Both qualitative and quantitative methods were employed to field-test the instrument. The two questionnaires were 
tested for the following purposes: 

1. To receive feedback on the questionnaires from a number of students in terms of clarity of their instructions, 

difficulty of their items, time needed for their administration, among others (as recommended by Creswell, 
2008); 

2. To discover the problems that the respondents may encounter while responding to the items (Liu, 2002); 




67 


ALLS 5(3):63-69, 2014 

3. To determine the reliability of the questionnaires through test-retest method (Dornyei, 2003); and 

4. As a corollary to these three purposes, to modify and refine the questionnaires (Dornyei, 2003). 

The field-test participants were a randomly selected sample of tertiary level students (n = 20). They were aged 19-24, 
and 70% of them were females. The two questionnaires were administered to these students twice with a time interval 
of two weeks for determining the reliability through test-retest method. This helped the researchers receive useful 
feedback and information on the clarity of instructions, the difficulty of items, the time needed for the instrument 
administration, the questionnaire layout, and for better administration. Following McBride (2007, p. 110), the students 
were required to write about their “general impressions” of the questionnaires, suggestions for improving them, and 
ideas about the instructions, difficulty level, time limit, and layout on a separate critique sheet attached to the 
questionnaires. Moreover, they were asked to circle the number of the survey items that they felt were difficult, unclear 
or ambiguous and were encouraged to make necessary changes for more clarity on the questionnaires. 

Before administration, the teacher was asked to present a listening activity to the students, “so that students would have 
a specific task on which to base their responses” (Vandergrift et al, 2006, p. 441). The first author then explained about 
the importance of the research and the need for the students to cooperate and respond carefully in order to make the 
results of the research more reliable. She then answered students' questions about the project. The students signed 
consent forms and were assured that all data collected would be kept anonymous. The questionnaires were distributed to 
the students. On average, it took them approximately 35 minutes to complete these procedures. Students were asked to 
double-check survey responses and return the questionnaires. The response rate was 100% and none of the completed 
questionnaires had to be discarded. 

8. Refining the instrument 

Administrating the questionnaires to the students provided some important information regarding the clarity and 
difficulty of the items, the time needed for administration, the layout, the necessity of some oral reminders, and the type 
of administration. This information was obtained both orally through the students’ questions or reactions, and in written 
form on the critique sheet. Table 4 summarizes these findings: 


Table 4. Learners’ feedback on LCPQ and LSUQ 




Feedback 

Modifications 


General 

Details 



Item clarity or 
difficulty 

The explanation related to the 
transactional listening should be 
highlighted or underlined. Some 
learners require oral explanation for 
it. 

The explanation was added 



Item 20 raises more than one 
question and sounds ambiguous. 

It was reworded. 

LCPQ 

Required 

administration 

time 

Students need a minimum of 10 
minutes to complete the two 
questionnaires. 

The recommended administration 
time was set at 15 minutes. 

Layout 

Suitable 

" 


Item clarity or 
difficulty 

Item 30 is difficult and raises more 
than one question. 

It was reworded. 



Examples should be used for item 5. 

An example was added. 



Highlight the important parts in 
items 27 and 28. 

The related parts were highlighted. 



Item 17 is confusing. 

It was reworded. 

a 

3 

Required 

administration 

time 

Students need a minimum of 10 
minutes to complete the two 
questionnaires. 

The recommended administration 
time was set at 15 minutes. 

t/2 

-J 

Layout 

Suitable 

- 


Since some students had not answered item 19 in LCPQ and items 3, 10, and 17 in LSUQ, these items were deleted 
after consultation with the experts. Indeed, the information to be elicited by these items could be obtained by other 
similar items. Consequently, the final number of items was 23 for LCPQ and 32 for LSUQ. 

Having fine-tuned the questionnaires, following the steps mentioned above, the researchers tested the internal reliability 
using SPSS (Versionl6). High Cronbach’s alpha indices of internal consistency were obtained for LCPQ (0.80) and 
LSUQ (0.87). 
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9. Conclusion 
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The present study sought to develop and validate an instrument, consisting of two questionnaires, to identify Iranian 
tertiary level EFL learners’ listening comprehension problems and strategies. The previous instruments, the available 
theory and the related literature were reviewed to construct the items. The questionnaires were validated by a panel of 
experts, based on whose feedback modifications were made to the questionnaires. This was followed by a test which 
provided useful information for the researchers to further fine-tune the instrument by observing how it works in the 
actual test setting. 

The paper is expected to have methodological implications, particularly for instrument developers in the area of 
language studies. The ground rules and the developmental procedure followed in this study can be considered in similar 
studies that aim at developing instruments. Researchers interested in the area of EFL listening skill may also find the 
instrument useful for testing EFL learners’ listening problems and strategies in their own learning context. The 
instrument may even be used in ESL learning context with some modifications. 

Likewise, language teachers can also benefit from the results of this study. The items of the present instrument can help 
teachers. They may use the instrument as a helpful checklist to be referred to in times they need to diagnose their 
learners’ listening comprehension problems or to find out which listening strategies their more successful learners use 
that distinguish them from less successful learners. 

A final implication of this study is that listening problems and strategies should not be approached in isolation. The 
instrument consists of a questionnaire that focuses on learners’ listening comprehension problems and another 
questionnaire which investigates their listening strategies. Combining the two constructs in one instrument means that 
learners’ listening strategies should not be taught or researched in isolation from their listening comprehension 
problems. 

Further research with a larger sample size will allow using more quantitative methods to validate the present instrument. 
The results of such validity tests will provide further empirical proof on the usefulness of the present instrument. 
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Appendix A: Original and revised items in LCPQ 


Original items 

Revised items 

28.1 get confused about the key ideas in the 
message. 

31.1 find it difficult to get the main ideas of what I 
hear. 

24. 1 find it difficult to get the supporting ideas of 
what 1 hear. 

2. There are too many words 1 cannot catch. 

2. There are too many words I cannot understand. 

11.1 am slow to recall the meaning of words that 
sound familiar. 

5. I am slow to remember the meaning of words 
that sound familiar. 

25.1 do not understand subsequent parts of input 
because of earlier problems. 

17. 1 do not understand later parts of text because 
of earlier problems. 

16.1 miss the next part of the material while 
thinking about the meaning. 

9. I miss the next part of the text while thinking 
about the meaning. 

25.1 do not understand subsequent parts of input 
because of earlier problems. 

17. 1 do not understand later parts of text because 
of earlier problems. 

29.1 find it difficult to get the details of the input. 

22.1 find it difficult to get the details of the text. 

23.1 find it difficult to follow unfamiliar topics or 
concepts. 

15.1 find it difficult to follow unfamiliar topics. 

5. The speaker does not speak loud enough. 

6. The speaker does not pause long enough. 

7. Iranian speakers are easier to understand than 

native speakers. 

Deleted 


Appendix B: Original and revised items in LSUQ 


Original items 

Revised items 

2.1 use the tone of voice (the rise and fall of voice) 
to guess the meaning of unknown words in an 
utterance. 

2. I use the tone of voice to guess the meaning of 
what 1 hear. 

13.1 listen to talk shows on the radio in the target 
language. 

13.1 watch TV in the target language. 

14.1 watch TV shows in the target language. 

12.1 listen to the radio in the target language. 

6. 1 use material in the response sheet to guess the 
meaning of unknown words. 

5. I use material in the answer sheet (e.g. the 
printed items, choices and pictures) to guess 
the meaning of what 1 hear. 

5. I use relationships between speakers in an oral 
text to guess the meaning of unknown words. 

5. I use relationships between speakers to guess the 
meaning what 1 hear. 

24. 1 decide to focus on the topic and ignore the 
distractors such as noise and others around me. 

23. 1 decide to focus on the topic and ignore the 
distractors such as people and things around 

me. 

6. 1 use material in the response sheet to guess the 
meaning of unknown words 

5. I use material in the answer sheet (e.g. the 
printed items, choices and pictures) to guess 
the meaning of what 1 hear. 






